docs/examples/Power Flow Example.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "89cf2628",
   "metadata": {},
   "source": [
    "# Power Flow Example\n",
    "\n",
    "In this notebook we will walk through an example of power flow calculation of `power-grid-model`. \n",
    "The following points are covered\n",
    " * Construction of the model\n",
    " * Run power flow once, and its relevant function arguments\n",
    " * Update (change) of the model\n",
    " * Run power flow in batch calculations, and its relevant function arguments\n",
    " * Error handling\n",
    "\n",
    "It serves as an example of how to use the Python API. For detailed API documentation, refer to\n",
    "[Python API reference](../api_reference/python-api-reference.md)\n",
    "and [Native Data Interface](../advanced_documentation/native-data-interface.md).\n",
    "\n",
    "\n",
    "## Example Network\n",
    "\n",
    "We use a simple network with 3 nodes, 1 source, 3 lines, and 2 loads. As shown below:\n",
    "\n",
    "```\n",
    " -----------------------line_8---------------\n",
    " |                                          |\n",
    "node_1 ---line_3--- node_2 ----line_5---- node_6\n",
    " |                    |                     |\n",
    "source_10          sym_load_4           sym_load_7\n",
    "```\n",
    "\n",
    "The 3 nodes are connected in a triangular way by 3 lines."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "ae11dc9a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# some basic imports\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "from power_grid_model import LoadGenType\n",
    "from power_grid_model import PowerGridModel, CalculationMethod, CalculationType\n",
    "from power_grid_model import initialize_array"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "f983cef7",
   "metadata": {},
   "source": [
    "## Input Dataset\n",
    "\n",
    "We create an input dataset by using the helper function `initialize_array`. \n",
    "- The units of all input are standard SI unit without any prefix, i.e. the unit of voltage is volt (V), not kV.\n",
    "- The attributes are of numpy array type.\n",
    "- `id` field is a unique field across the `input_data` dictionary. The keys of `input-data` are predefined.\n",
    "\n",
    "Please refer to [Components](../user_manual/components.md) for detailed explanation of all component types and their input/output attributes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "6f008736",
   "metadata": {},
   "outputs": [],
   "source": [
    "# node\n",
    "node = initialize_array(\"input\", \"node\", 3)\n",
    "node[\"id\"] = np.array([1, 2, 6])\n",
    "node[\"u_rated\"] = [10.5e3, 10.5e3, 10.5e3]\n",
    "\n",
    "# line\n",
    "line = initialize_array(\"input\", \"line\", 3)\n",
    "line[\"id\"] = [3, 5, 8]\n",
    "line[\"from_node\"] = [1, 2, 1]\n",
    "line[\"to_node\"] = [2, 6, 6]\n",
    "line[\"from_status\"] = [1, 1, 1]\n",
    "line[\"to_status\"] = [1, 1, 1]\n",
    "line[\"r1\"] = [0.25, 0.25, 0.25]\n",
    "line[\"x1\"] = [0.2, 0.2, 0.2]\n",
    "line[\"c1\"] = [10e-6, 10e-6, 10e-6]\n",
    "line[\"tan1\"] = [0.0, 0.0, 0.0]\n",
    "line[\"i_n\"] = [1000, 1000, 1000]\n",
    "\n",
    "# load\n",
    "sym_load = initialize_array(\"input\", \"sym_load\", 2)\n",
    "sym_load[\"id\"] = [4, 7]\n",
    "sym_load[\"node\"] = [2, 6]\n",
    "sym_load[\"status\"] = [1, 1]\n",
    "sym_load[\"type\"] = [LoadGenType.const_power, LoadGenType.const_power]\n",
    "sym_load[\"p_specified\"] = [20e6, 10e6]\n",
    "sym_load[\"q_specified\"] = [5e6, 2e6]\n",
    "\n",
    "# source\n",
    "source = initialize_array(\"input\", \"source\", 1)\n",
    "source[\"id\"] = [10]\n",
    "source[\"node\"] = [1]\n",
    "source[\"status\"] = [1]\n",
    "source[\"u_ref\"] = [1.0]\n",
    "\n",
    "# all\n",
    "input_data = {\n",
    "    \"node\": node,\n",
    "    \"line\": line,\n",
    "    \"sym_load\": sym_load,\n",
    "    \"source\": source\n",
    "}"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "682c1c48",
   "metadata": {},
   "source": [
    "**We can print the input dataset by converting the numpy array to dataframe.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "f0c8c3e8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   id  node  status  type  p_specified  q_specified\n",
      "0   4     2       1     0   20000000.0    5000000.0\n",
      "1   7     6       1     0   10000000.0    2000000.0\n"
     ]
    }
   ],
   "source": [
    "print(pd.DataFrame(input_data[\"sym_load\"]))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6e332961",
   "metadata": {},
   "source": [
    "**(Assignment 1)**"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "68a38330",
   "metadata": {},
   "source": [
    "## Validation (optional)\n",
    "For efficiency reasons, most of the data is not explicitly validated in the power grid model. However, in most cases, a power flow calculation will fail/crash if the data is invalid. Often with a low level error message that is hard to relate to the original objects. Therfore, it is recommended to always validate your data before constructing a PowerGridModel instance.\n",
    "\n",
    "The simplest and most effective way to validate your data is by using `assert_valid_input_data()` which will throw an error if it encounters any invalid data. See [Validation Examples](./Validation%20Examples.ipynb) for more detailed information on the validation functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "40509eaf",
   "metadata": {},
   "outputs": [],
   "source": [
    "from power_grid_model.validation import assert_valid_input_data\n",
    "assert_valid_input_data(input_data=input_data, calculation_type=CalculationType.power_flow)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c0b675b2",
   "metadata": {},
   "source": [
    "**(Assignment 2)**"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "afae0224",
   "metadata": {},
   "source": [
    "## Construction\n",
    "\n",
    "The construction of the model is just calling the constructor of `PowerGridModel`. The default `system_frequency` is 50.0 Hz.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "7ef134e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = PowerGridModel(input_data)\n",
    "# model = PowerGridModel(input_data, system_frequency=60.0)  # if you have another system frequency"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d87976c9",
   "metadata": {},
   "source": [
    "**(Assignment 3)**"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "e3605c3e",
   "metadata": {},
   "source": [
    "## One-time Power Flow Calculation\n",
    "\n",
    "You can call the method `calculate_power_flow` to do a one-time calculation based on the current network data in the model. In this case you should not specify the argument `update_data` as it is used for batch calculation.\n",
    "\n",
    "The following command executes a one-time power flow calculation with (they are also the default values for those arguments)\n",
    "* Symmetric calculation\n",
    "* Newton-Raphson method\n",
    "* Error tolerance: 1e-8\n",
    "* Maximum number of iteration: 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "44c2de63",
   "metadata": {},
   "outputs": [],
   "source": [
    "output_data = model.calculate_power_flow(\n",
    "    symmetric=True,\n",
    "    error_tolerance=1e-8,\n",
    "    max_iterations=20,\n",
    "    calculation_method=CalculationMethod.newton_raphson)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d08aaf45",
   "metadata": {},
   "source": [
    "### Result Dataset\n",
    "\n",
    "We can also print a result dataset of node and line by converting the array to dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "a581a36e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "------node result------\n",
      "   id  energized      u_pu             u   u_angle             p             q\n",
      "0   1          1  0.998988  10489.375043 -0.003039  3.121451e+07  6.991358e+06\n",
      "1   2          1  0.952126   9997.325181 -0.026031 -2.000000e+07 -5.000000e+06\n",
      "2   6          1  0.962096  10102.012975 -0.021895 -1.000000e+07 -2.000000e+06\n",
      "------line result------\n",
      "   id  energized   loading        p_from        q_from      i_from  \\\n",
      "0   3          1  0.985666  1.736010e+07  4.072097e+06  981.460041   \n",
      "1   5          1  0.205940 -3.365614e+06 -1.178649e+06  205.939917   \n",
      "2   8          1  0.783206  1.385441e+07  2.919262e+06  779.311446   \n",
      "\n",
      "         s_from          p_to          q_to        i_to          s_to  \n",
      "0  1.783129e+07 -1.663439e+07 -3.821351e+06  985.666324  1.706768e+07  \n",
      "1  3.566030e+06  3.396558e+06  8.861080e+05  200.617323  3.510241e+06  \n",
      "2  1.415863e+07 -1.339656e+07 -2.886108e+06  783.206396  1.370392e+07  \n"
     ]
    }
   ],
   "source": [
    "print(\"------node result------\")\n",
    "print(pd.DataFrame(output_data[\"node\"]))\n",
    "print(\"------line result------\")\n",
    "print(pd.DataFrame(output_data[\"line\"]))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "24893218",
   "metadata": {},
   "source": [
    "### Select component types in the result dataset\n",
    "\n",
    "By default `power-grid-model` calculates the result attributes for all components. If you do not need all the component types in your results, for example if you are only interested in `node`, you can specify the list of the component types you want in the result dataset in the argument `output_component_types`.\n",
    "\n",
    "The following code calculate the power flow and only generate the output attributes for `node`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "30fdf18a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "List of component types in result dataset\n",
      "['node']\n",
      "------node result------\n",
      "   id  energized      u_pu             u   u_angle             p             q\n",
      "0   1          1  0.998988  10489.375043 -0.003039  3.121451e+07  6.991358e+06\n",
      "1   2          1  0.952126   9997.325181 -0.026031 -2.000000e+07 -5.000000e+06\n",
      "2   6          1  0.962096  10102.012975 -0.021895 -1.000000e+07 -2.000000e+06\n"
     ]
    }
   ],
   "source": [
    "output_data = model.calculate_power_flow(\n",
    "    symmetric=True,\n",
    "    error_tolerance=1e-8,\n",
    "    max_iterations=20,\n",
    "    calculation_method=CalculationMethod.newton_raphson,\n",
    "    output_component_types=[\"node\"])\n",
    "\n",
    "print(\"List of component types in result dataset\")\n",
    "print(list(output_data.keys()))\n",
    "print(\"------node result------\")\n",
    "print(pd.DataFrame(output_data[\"node\"]))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "fc428870",
   "metadata": {},
   "source": [
    "### Calculation Method\n",
    "\n",
    "There are four calculation methods available it is defined as enumrations:\n",
    "* **Iterative methods**: These Iterative methods converge to the true solution. You specify the `error_tolerance` and `max_iterations` as the iteration stopping condition. If the maximum number of iteration is reached and the error is still higher than the error tolerance, an exception will be thrown.\n",
    "    * `CalculationMethod.newton_raphson`: traditional Newton-Raphson method.\n",
    "    * `CalculationMethod.iterative_current`: a novel jacobi like method.\n",
    "* **Linear methods**: linearization approximation methods. In this case the `error_tolerance` and `max_iterations` are not relevant here. They are also garanteed to have a solution.\n",
    "    * `CalculationMethod.linear`: It treats all loads as constant impedances.\n",
    "    * `CalculationMethod.linear_current`: It is essentially a single iteration of Iterative current."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "92c0e956",
   "metadata": {},
   "source": [
    "**Following command calculates the power flow with linear method. Note the difference of result for node compared to the results from Newton-Raphson method.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "ecbb8eeb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "------node result------\n",
      "   id  energized      u_pu             u   u_angle             p             q\n",
      "0   1          1  0.999087  10490.414234 -0.002789  2.862536e+07  6.264884e+06\n",
      "1   2          1  0.956325  10041.412991 -0.023873 -1.829115e+07 -4.572788e+06\n",
      "2   6          1  0.965236  10134.973932 -0.020204 -9.316798e+06 -1.863360e+06\n"
     ]
    }
   ],
   "source": [
    "output_data = model.calculate_power_flow(symmetric=True, calculation_method=CalculationMethod.linear)\n",
    "print(\"------node result------\")\n",
    "print(pd.DataFrame(output_data[\"node\"]))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c21d79b4",
   "metadata": {},
   "source": [
    "**(Assignment 4)**"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "bdaa7fae",
   "metadata": {},
   "source": [
    "## Update Model\n",
    "\n",
    "One can mutate the model by providing a dictionary of update data. This is similar to creating an input dataset. The only difference is to initialize the array with the `update` key.\n",
    "\n",
    "Refer to [Components](../user_manual/components.md) for detailed explanation of all mutable attributes per component type.\n",
    "\n",
    "NOTE: the initialized the array contains default NULL values for everything. If you do not need to update an attribute, you do not have to specify it. In the C++ core, the default NULL value will be treated as \"not updating\" for this attribute.\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "929824c4",
   "metadata": {},
   "source": [
    "### Create Update Dataset\n",
    "The following code creates an update dataset which changes the two loads and switches Line #3 off at the from side.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "1d035c65",
   "metadata": {},
   "outputs": [],
   "source": [
    "update_sym_load = initialize_array(\"update\", \"sym_load\", 2)\n",
    "update_sym_load[\"id\"] = [4, 7]  # same ID\n",
    "update_sym_load[\"p_specified\"] = [30e6, 15e6]  # change active power\n",
    "# leave reactive power the same, no need to specify\n",
    "\n",
    "update_line = initialize_array(\"update\", \"line\", 1)\n",
    "update_line[\"id\"] = [3]  # change line ID 3\n",
    "update_line[\"from_status\"] = [0]  # switch off at from side\n",
    "# leave to-side swiching status the same, no need to specify\n",
    "\n",
    "update_data = {\n",
    "    \"sym_load\": update_sym_load,\n",
    "    \"line\": update_line\n",
    "}"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "3d9d3d45",
   "metadata": {},
   "source": [
    "### Validation (optional)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "63ea4cc3",
   "metadata": {},
   "outputs": [],
   "source": [
    "from power_grid_model.validation import assert_valid_batch_data\n",
    "assert_valid_batch_data(input_data=input_data, update_data=update_data, calculation_type=CalculationType.power_flow)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "48872e8d",
   "metadata": {},
   "source": [
    "### Call Update Method\n",
    "\n",
    "Call the update function on a copy of the model. Note that you don't need to make a copy in general, this is only done to keep the original model intact."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "188f6663",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_2 = model.copy()\n",
    "model_2.update(update_data=update_data)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "ce9af070",
   "metadata": {},
   "source": [
    "### Re-calculate Power Flow\n",
    "\n",
    "We calculate power flow again with all default arguments. And print the line results. See the difference of the updated data. Line #3 has zero power flow at the from-side and very small power flow at to-side (due to capacitance)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "a93c1e16",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "------line result------\n",
      "   id  energized   loading        p_from        q_from       i_from  \\\n",
      "0   3          1  0.014030  0.000000e+00  0.000000e+00     0.000000   \n",
      "1   5          1  2.268159 -3.000004e+07 -4.812057e+06  2268.158655   \n",
      "2   8          1  3.249928  5.676867e+07  1.571464e+07  3245.829906   \n",
      "\n",
      "         s_from          p_to          q_to         i_to          s_to  \n",
      "0  0.000000e+00  3.691991e+01 -1.879430e+05    14.030122  1.879430e+05  \n",
      "1  3.038352e+07  3.385470e+07  7.678867e+06  2265.269969  3.471463e+07  \n",
      "2  5.890358e+07 -4.885470e+07 -9.678867e+06  3249.927645  4.980424e+07  \n"
     ]
    }
   ],
   "source": [
    "output_data = model_2.calculate_power_flow()\n",
    "print(\"------line result------\")\n",
    "print(pd.DataFrame(output_data[\"line\"]))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "4f59d0f5",
   "metadata": {},
   "source": [
    "## Batch Calculation\n",
    "\n",
    "We can use the same method `calculate_power_flow` to calculate a number of scenarios in one go. To do this, you need to supply an `update_data` argument. This argument contains a dictionary of 2D update arrays (one array per component type).\n",
    "\n",
    "The model uses the current data as the base scenario. For each individual calculation, the model applies each mutation to the base scenario and calculates the power flow.\n",
    "\n",
    "**NOTE: after the batch calculation, the original model will be kept unchanged. Internally the program copies the original model to multiple batch models for the calculation.**\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "56850333",
   "metadata": {},
   "source": [
    "### Independent Batch Dataset\n",
    "\n",
    "There are two ways to specify the mutations.\n",
    "\n",
    "* For each scenario only specify the objects that are changed in this scenario.\n",
    "* For each scenario specify all objects that are changed in one or more scenarios.\n",
    "\n",
    "The latter is called independent batch dataset. Because all relevant objects are specified in each batch, different choices regarding performance optimization may be made in either case.\n",
    "\n",
    "In general, the following is advised:\n",
    "\n",
    "* Use the non-independent batch dataset approach whenever few parameters change per scenario, but the batch samples many different components, e.g. during N-1 tests.\n",
    "* Use the independent batch dataset approach when a dense sampling of the parameter space is desired for relatively a few different components, e.g. during time series power flow calculation\n",
    "\n",
    "See also [performance guide](../user_manual/performance-guide.md) for the latest recommendations."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "7dd1f621",
   "metadata": {},
   "source": [
    "### Examples\n",
    "\n",
    "Below we show three examples."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "93a22edb",
   "metadata": {},
   "source": [
    "#### Time Series Profile\n",
    "\n",
    "The following code creates a load profile with 10 timestamps for the two loads. The two loads are always present for all mutation scenarios. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "42d5cd8f",
   "metadata": {},
   "outputs": [],
   "source": [
    "load_profile = initialize_array(\n",
    "    \"update\", \"sym_load\", (10, 2)\n",
    ")  # note the shape of the array, 10 scenarios, 2 objects (loads)\n",
    "# below is an assignment of shape (1, 2) array to shape (10, 2) array\n",
    "# the numpy broadcasting rule ensures that the same object ids are repeated 10 times\n",
    "# therefore the two objects are present for all the scenarios\n",
    "load_profile[\"id\"] = [[4, 7]]\n",
    "# this is a scale of load from 0% to 100%\n",
    "# the array is an (10, 2) shape, each row is a scenario, each column is an object\n",
    "load_profile[\"p_specified\"] = [[30e6, 15e6]] * np.linspace(0, 1, 10).reshape(-1, 1)\n",
    "\n",
    "time_series_mutation = {\"sym_load\": load_profile}\n",
    "\n",
    "assert_valid_batch_data(input_data=input_data, update_data=time_series_mutation, calculation_type=CalculationType.power_flow)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "363aa67d",
   "metadata": {},
   "source": [
    "We can calculate the time series and print the current of the lines."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "4e25006f",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 193.06162675   64.92360593  137.59086941]\n",
      " [ 248.65360093   72.28087746  185.86691646]\n",
      " [ 368.12834615   90.73999329  285.46275837]\n",
      " [ 510.20016036  115.51167381  401.15982574]\n",
      " [ 662.04311447  143.73983633  523.57045909]\n",
      " [ 819.63118685  174.08971222  649.95104138]\n",
      " [ 981.46004118  205.93991655  779.31144601]\n",
      " [1146.90787963  238.98921914  911.25418138]\n",
      " [1315.7236725   273.08968816 1045.6266953 ]\n",
      " [1487.83778526  308.17354058 1182.39546494]]\n"
     ]
    }
   ],
   "source": [
    "output_data = model.calculate_power_flow(update_data=time_series_mutation)\n",
    "print(output_data[\"line\"][\"i_from\"])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c9d23a92",
   "metadata": {},
   "source": [
    "##### Accessing batch data\n",
    "\n",
    "It may be a bit unintuitive to read the `output_data` or `update_data` of a component directly as they are a dictionary of 3 dimension data, ie. $ids \\times batches \\times  attributes$. Remember that the `output_data` or `update_data` are a dictionary of numpy structured arrays. Hence the component should be indexed first. The index that follows can be [indexed with numpy structured arrays](https://numpy.org/doc/stable/user/basics.rec.html#indexing-structured-arrays)\n",
    "\n",
    "To read the result of a single batch, lets say 1st batch,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "afccf7a2",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   id  energized   loading        p_from        q_from      i_from  \\\n",
      "0   3          1  0.212031  29511.239220  3.508895e+06  193.061627   \n",
      "1   5          1  0.064924  -1267.160636 -1.172098e+06   64.923606   \n",
      "2   8          1  0.156578  19805.345039  2.500724e+06  137.590869   \n",
      "\n",
      "         s_from         p_to          q_to        i_to          s_to  \n",
      "0  3.509019e+06  1267.160636 -3.827902e+06  212.030847  3.827902e+06  \n",
      "1  1.172099e+06  3574.919119  8.320006e+05   46.000611  8.320083e+05  \n",
      "2  2.500802e+06 -3574.919119 -2.832001e+06  156.577603  2.832003e+06  \n"
     ]
    }
   ],
   "source": [
    "print(pd.DataFrame(output_data[\"line\"][0]))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6d9f285a",
   "metadata": {},
   "source": [
    "Or maybe we wish to find result of a single component, (eg. 1st line) in all batches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "9af1be38",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 193.06162675  248.65360093  368.12834615  510.20016036  662.04311447\n",
      "  819.63118685  981.46004118 1146.90787963 1315.7236725  1487.83778526]\n"
     ]
    }
   ],
   "source": [
    "print(output_data[\"line\"][\"i_from\"][:,0])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "8aeb59b2",
   "metadata": {},
   "source": [
    "**(Assignment 5)**"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "ac217192",
   "metadata": {},
   "source": [
    "#### N-1 Scenario while specifying all objects in each scenario\n",
    "\n",
    "The following code creates a N-1 scenario for all three lines. There are 3 scenarios, in each scenario, the from/to status of one line is switched off. However, all three lines are present in all mutation dataset. This means that for each scenario you will update two lines with the same switching status.\n",
    "\n",
    "Specifying all objects can bring computational advantages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "041368dc",
   "metadata": {},
   "outputs": [],
   "source": [
    "line_profile = initialize_array(\"update\", \"line\", (3, 3))  # 3 scenarios, 3 objects (lines)\n",
    "# below the same broadcasting trick\n",
    "line_profile[\"id\"] = [[3, 5, 8]]\n",
    "# fully specify the status of all lines, even it is the same as the base scenario\n",
    "line_profile[\"from_status\"] = [\n",
    "    [0, 1, 1],\n",
    "    [1, 0, 1],\n",
    "    [1, 1, 0]\n",
    "]\n",
    "line_profile[\"to_status\"] = [\n",
    "    [0, 1, 1],\n",
    "    [1, 0, 1],\n",
    "    [1, 1, 0]\n",
    "]\n",
    "\n",
    "n_min_1_mutation_update_all = {\"line\": line_profile}\n",
    "\n",
    "assert_valid_batch_data(input_data=input_data, update_data=n_min_1_mutation_update_all, calculation_type=CalculationType.power_flow)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "8e2b8867",
   "metadata": {},
   "source": [
    "We can calculate the N-1 and print the current of the lines. It is clear that per scenario one line is disabled."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "34338ce3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[   0.         1352.02947002 1962.69857764]\n",
      " [1199.97577809    0.          573.32693369]\n",
      " [1877.3560102   634.81512055    0.        ]]\n"
     ]
    }
   ],
   "source": [
    "output_data = model.calculate_power_flow(update_data=n_min_1_mutation_update_all)\n",
    "print(output_data[\"line\"][\"i_from\"])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "4238fc30",
   "metadata": {},
   "source": [
    "#### N-1 Scenario where only the changed objects are specified\n",
    "\n",
    "The following code creates a N-1 scenario for all three lines. There are 3 scenarios, in each scenario, the from/to status of one line is switched off. In this dataset we only specify one line per mutation. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "04e56690",
   "metadata": {},
   "outputs": [],
   "source": [
    "line_profile = initialize_array(\"update\", \"line\", (3, 1))  # 3 scenarios, 1 object mutation per scenario\n",
    "# for each mutation, only one object is specified\n",
    "line_profile[\"id\"] = [[3], [5], [8]]\n",
    "# specify only the changed status (switch off) of one line\n",
    "line_profile[\"from_status\"] = [[0], [0], [0]]\n",
    "line_profile[\"to_status\"] = [[0], [0], [0]]\n",
    "\n",
    "n_min_1_mutation_update_specific = {\"line\": line_profile}\n",
    "\n",
    "assert_valid_batch_data(input_data=input_data, update_data=n_min_1_mutation_update_specific, calculation_type=CalculationType.power_flow)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "7365ab26",
   "metadata": {},
   "source": [
    "We can calculate the N-1 and print the current of the lines. Note the results are the same as the independent N-1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "0d5b94c2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[   0.         1352.02947002 1962.69857764]\n",
      " [1199.97577809    0.          573.32693369]\n",
      " [1877.3560102   634.81512055    0.        ]]\n"
     ]
    }
   ],
   "source": [
    "output_data = model.calculate_power_flow(update_data=n_min_1_mutation_update_specific)\n",
    "print(output_data[\"line\"][\"i_from\"])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "9ec38cc0",
   "metadata": {},
   "source": [
    "### Caching Topology\n",
    "\n",
    "* If a batch scenario does not change the switching status of branches and sources, the model will re-use the pre-built internal graph/matrices for each calculation. Time-series load profile calculation is a typical use case. This can bring performance benefits.\n",
    "* If a batch scenario changes the switching status of branches and sources, the topology changes and no caching can be done for that scenario. N-1 check is a typical use case.\n",
    "* See the [Performance Guide](../user_manual/performance-guide.md) for tips and tricks regarding performance."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6c2ee42e",
   "metadata": {},
   "source": [
    "### Parallel Computing\n",
    "\n",
    "The batch calculation supports shared memory multi-threading parallel computing. The common internal states and variables are shared as much as possible to save memory usage and avoid copy. In the C++ implementation `std::shared_ptr` of `const` variable is used to share the resources between threads (which will not mutate the shared data).\n",
    "\n",
    "You can set `threading` parameter to enable/disable parallel computing.\n",
    "* `threading=-1`, use sequential computing (default)\n",
    "* `threading=0`, use number of threads available from the machine hardware (recommended)\n",
    "* `threading>0`, set the number of threads you want to use"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "ab485680",
   "metadata": {},
   "source": [
    "**(Assignment 6)**"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "da903865",
   "metadata": {},
   "source": [
    "## Error Handling\n",
    "\n",
    "The error handeling of `power-grid-model` is using exceptions in all cases."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "cd824d12",
   "metadata": {},
   "source": [
    "### Raise Exception\n",
    "\n",
    "If there is an error in the ctypes wrapper, a relevant exception will be raised (e.g. `KeyError`). If there is an error in the C++ core, the C++ exception will be translated into a `PowerGridError` in Python. Below we show some examples for the errors in the construction, update, and calculation."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "f94eae05",
   "metadata": {},
   "source": [
    "#### Construction\n",
    "\n",
    "We try to construct a grid with a line connecting two nodes with different rated voltages. An error is raised in the construction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "b5f10bae",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Conflicting voltage for line 3\n",
      " voltage at from node 1 is 10500.000000\n",
      " voltage at to node 2 is 150000.000000\n",
      "\n",
      "Try validate_input_data() or validate_batch_data() to validate your data.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from power_grid_model.errors import PowerGridError\n",
    "\n",
    "# node\n",
    "node_error = initialize_array(\"input\", \"node\", 2)\n",
    "node_error[\"id\"] = [1, 2]\n",
    "node_error[\"u_rated\"] = [10.5e3, 150.0e3]  # different rated voltages\n",
    "# line\n",
    "line_error = initialize_array(\"input\", \"line\", 1)\n",
    "line_error[\"id\"] = [3]\n",
    "line_error[\"from_node\"] = [1]\n",
    "line_error[\"to_node\"] = [2]\n",
    "line_error[\"from_status\"] = [1]\n",
    "line_error[\"to_status\"] = [1]\n",
    "line_error[\"r1\"] = [0.25]\n",
    "line_error[\"x1\"] = [0.2]\n",
    "line_error[\"c1\"] = [10e-6]\n",
    "line_error[\"tan1\"] = [0.0]\n",
    "line_error[\"i_n\"] = [1000]\n",
    "\n",
    "try:\n",
    "    PowerGridModel(\n",
    "        {\n",
    "            \"node\": node_error,\n",
    "            \"line\": line_error\n",
    "        }\n",
    "    )\n",
    "except PowerGridError as e:\n",
    "    print(e)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "80a46ba7",
   "metadata": {},
   "source": [
    "#### Update\n",
    "\n",
    "We try to update the model with a non-existing line id."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "1a221507",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The id cannot be found: 12345\n",
      "\n",
      "Try validate_input_data() or validate_batch_data() to validate your data.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "line_update_error = initialize_array(\"update\", \"line\", 1)\n",
    "line_update_error[\"id\"] = [12345]  # non-existing\n",
    "line_update_error[\"from_status\"] = [1]\n",
    "\n",
    "\n",
    "try:\n",
    "    model.update(update_data={\"line\": line_update_error})\n",
    "except PowerGridError as e:\n",
    "    print(e)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "b6ff32c2",
   "metadata": {},
   "source": [
    "#### Error in Calculation\n",
    "\n",
    "Following command calculates the power flow with Newton-Raphson method that fails to converge, because the error tolerance is too low. It throws a `PowerGridError`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "541af620",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration failed to converge after 20 iterations! Max deviation: 0.000000, error tolerance: 0.000000.\n",
      "\n",
      "Try validate_input_data() or validate_batch_data() to validate your data.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    model.calculate_power_flow(\n",
    "        symmetric=True,\n",
    "        error_tolerance=1e-20,\n",
    "        max_iterations=20,\n",
    "        calculation_method=CalculationMethod.newton_raphson)\n",
    "except PowerGridError as e:\n",
    "    print(e)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "97c5f35a",
   "metadata": {},
   "source": [
    "#### Error in Batch Calculation\n",
    "\n",
    "Because parallel computing is allowed in batch calculation. The program will be terminated if an exception is thrown from the threads. Therefore, all the C++ exceptions are caught within the thread, and the error messages are re-thrown in the main thread and propagate as `PowerGridBatchError` in the Python side.\n",
    "\n",
    "There could happen that only some of the scenarios fail in a batch calculation and others succeed. You can choose to specify the argument `continue_on_batch_error` (default `False`) if an error should be thrown in this case or let the program continue with partially valid results."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "1cb653a1",
   "metadata": {},
   "source": [
    "We modify the time-series load profile to add some errors. We add an unknown `id` for scenario 3, and a too large power in scenario 7.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "20d8285c",
   "metadata": {},
   "outputs": [],
   "source": [
    "time_series_mutation[\"sym_load\"][\"id\"][3] = 1000  # unknown id\n",
    "time_series_mutation[\"sym_load\"][\"p_specified\"][7] = 1e100  # large power\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "b702eb15",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are errors in the batch calculation.\n",
      "Try validate_input_data() or validate_batch_data() to validate your data.\n",
      "\n",
      "Failed scenarios: [3 7]\n",
      "Succeeded scenarios: [0 1 2 4 5 6 8 9]\n",
      "Error messages: ['The id cannot be found: 1000\\n', 'Iteration failed to converge after 20 iterations! Max deviation: 43691065954360128432752954469939668333328459052496602949289781148532517437499693334528.000000, error tolerance: 0.000000.\\n']\n"
     ]
    }
   ],
   "source": [
    "# we run the batch calculation with default, it will raise error\n",
    "\n",
    "from power_grid_model.errors import PowerGridBatchError\n",
    "\n",
    "try:\n",
    "    output_data = model.calculate_power_flow(\n",
    "        update_data=time_series_mutation,\n",
    "    )\n",
    "except PowerGridBatchError as e:\n",
    "    print(e)\n",
    "    print(f\"Failed scenarios: {e.failed_scenarios}\")\n",
    "    print(f\"Succeeded scenarios: {e.succeeded_scenarios }\")\n",
    "    print(f\"Error messages: {e.error_messages }\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "1ba71901",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Node data with invalid results\n",
      "[[0.99940117 0.99268579 0.99452137]\n",
      " [0.99934769 0.98622639 0.98935286]\n",
      " [0.99928838 0.97965401 0.98409554]\n",
      " [0.         0.         0.        ]\n",
      " [0.99915138 0.96614948 0.97329879]\n",
      " [0.99907317 0.95920586 0.96775071]\n",
      " [0.9989881  0.95212621 0.96209647]\n",
      " [0.         0.         0.        ]\n",
      " [0.99879613 0.93753005 0.95044796]\n",
      " [0.9986885  0.92999747 0.94444167]]\n",
      "Node data with only valid results\n",
      "[[0.99940117 0.99268579 0.99452137]\n",
      " [0.99934769 0.98622639 0.98935286]\n",
      " [0.99928838 0.97965401 0.98409554]\n",
      " [0.99915138 0.96614948 0.97329879]\n",
      " [0.99907317 0.95920586 0.96775071]\n",
      " [0.9989881  0.95212621 0.96209647]\n",
      " [0.99879613 0.93753005 0.95044796]\n",
      " [0.9986885  0.92999747 0.94444167]]\n"
     ]
    }
   ],
   "source": [
    "# we run the batch calculation with continue_on_batch_error=True, \n",
    "#    it will return the results with partially valid data\n",
    "\n",
    "\n",
    "output_data = model.calculate_power_flow(\n",
    "    update_data=time_series_mutation,\n",
    "    continue_on_batch_error=True\n",
    ")\n",
    "\n",
    "# print node data for u_pu, note that the data is rubbish for scenario 3 and 7\n",
    "print(\"Node data with invalid results\")\n",
    "print(output_data[\"node\"][\"u_pu\"])\n",
    "\n",
    "# we can print the data with only succeeded scenarios\n",
    "e = model.batch_error\n",
    "print(\"Node data with only valid results\")\n",
    "print(output_data[\"node\"][\"u_pu\"][e.succeeded_scenarios])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "74994363",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.1"
  },
  "vscode": {
   "interpreter": {
    "hash": "f2534ebf7a1a13ecc51c3a04ce741b49cf1feb97b5ca55170ed7b2036b4058c0"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}